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SECTION  I 
INTRODUCTION 

The  Training  Analysis  and  Evaluation  Group  (TAEG)  was  tasked  by  the 
Chief  of  Naval  Education  and  Training  (CNET)  to  conduct  a  study  of  the  long- 
range  role  and  mission  of  the  Instructional  Program  Development  Centers 
(IPDCs).*  The  tasking  requested  TAEG  to  review  the  original  "A  Plan  for  the 
Centralized  Management  of  Instructional  Systems  Development  (ISD)  Within  the 
Naval  Education  and  Training  Command"*  (conmonly  referred  to  as  the  "Blue 
Book")  and  assess  progress  to  date.  It  was  further  requested  that  the 
concept  of  the  IPDC  network  be  analyzed  and  extended  to  include  all  activity 
relating  to  "A,"  "C,"  and  11 F"  Schools  under  CNET  cognizance.  The  effort 
envisaged  included  assessing  IPDC  and  school  development  capabilities  and 
developing  recommendations  for  integrating  these  capabilities  into  a  coordin¬ 
ated  command-wide  I PD  program. 

A  two  phased  study  was  initiated.  Phase  I  presented  an  analysis  of  IPD 
productivity  from  July  1976  to  September  1980  which  covers  the  period  from 
initial  activity  at  IPDC  San  Diego  to  the  latest  accounting  period  prior  to 
report  preparation.  Productivity  data  for  IPD  activity  relating  to  28  courses 
were  examined  in  detail  for  this  period  and  recommendations  for  productivity 
improvement  were  proposed.  The  results  of  this  effort  were  published  by 
Swope  and  Keeler  (1981). 

Phase  II  assessed  current  IPD  management  and  operational  practices  and 
examined  existing  organizational  structures.  Based  upon  this  assessment  and 
an  in-depth  study  of  productivity  measures,  a  number  of  recomnendati ons  for 
enhancing  IPD  productivity  were  developed.  The  present  report  describes  this 
effort. 

PURPOSE 

The  purpose  of  the  present  study  is  to: 

1.  identify  functional  areas  within  the  IPD  program  to  which  produc¬ 
tivity  enhancement  measures  can  be  applied 

2.  make  recommendations  for  implementation  and  provide  supporting 
rationale  for  the  productivity  measures  identified 

3.  organize  the  productivity  enhancement  recomnendati ons  in  a  format 
suitable  for  use  as  a  data  base  to  support  the  development  and  implementation 
of  a  long-range  CNET  IPD  plan. 


^NET  N-53  ltr  of  21  Feb  1980,  subj:  Tasking  for  a  study  of  the  long-range 
role  and  mission  of  Instructional  Program  Development  Centers  (IPDC's). 

? 

Chief  of  Naval  Education  and  Training  Support  memorandum  01A:EIL  of 
11  August  1975,  subj:  Plan  for  the  Implementation  of  ISD  in  NAVEDTRACOM. 
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SECTION  II 

BACKGROUND  AND  APPROACH 


BACKGROUND 

3 

A  study  group  was  established  in  January  1975  by  the  then  existing 
Chief  of  Naval  Education  and  Training  Support  (CNETS)  to  develop  a  concept  of 
operations  for  implementing  a  centrally  managed  Instructional  Systems  Deve¬ 
lopment  (ISD)  program.  The  concept  of  operations  developed  by  the  study 
group  was  presented  to  the  CNET  during  May  1975.  He  judged  the  plan  of 
centralizing  ISD  at  one  location  inefficient.  The  CNET  subsequently  provided 
guidance  which  called  for  revising  the  initial  concept.  This  guidance  sug¬ 
gested  centralizing  ISD  management  and  decentralizing  development.  Based 
upon  this  guidance,  CNETS  reconvened  the  study  group  to  develop  a  revised 
plan  which  would  provide  for  dispersion  of  the  program  development  function 
while  maintaining  effective  centralized  program  management.  "A  Plan  for  the 
Centralized  Management  of  Instructional  Systems  Development  (ISD)  Within  the 
Naval  Education  and  Training  Command"  was  developed  in  accordance  with  the 
revised  guidance  and  was  submitted  to  CNET  for  approval  in  August  1975.  The 
plan  was  approved  in  concept  by  CNET  in  September  1975  and  implementation  was 
directed.  This  plan  then  became  the  basis  for  implementing  the  ISD  program. 

The  "Blue  Book,"  although  well  conceived  in  1975,  has  become  outdated. 
The  TAEG  was  tasked  to  reexamine  the  long-range  role  of  IPDCs  in  support  of 
the  CNET  command-wide  IPD  mission.  In  addition,  the  CNET  Assistant  Chief  of 
Staff  for  Training  Systems  Management  expressed  a  desire  to  develop  an  up¬ 
dated  long-range  command-wide  IPD  plan.  The  recommendations  proposed  in  this 
study  are  intended  to  support  this  planning  effort. 

APPROACH 

A  three-step  process  was  employed  in  developing  this  study. 

Initially,  a  data  base  was  developed  utilizing  inputs  from  current  ISD 
directives  and  instructions  and  recently  published  CNET/TAEG  documents.  The 
documents  having  the  greatest  relevancy  durinq  this  development  are  Hodak, 
Middleton,  and  Rankin  (1979);  Swope  and  Keeler  (1981);  and  NAVEDTRA  110 
(1978).  Next,  interviews  with  key  IPD  personnel  at  both  the  management  and 
operations  level  provided  a  substantial  range  of  information.  Information 
gathered  from  about  60  IPD  practitioners  provided  necessary  current  expert 
opinion  in  a  real-world  setting.  Finally,  a  framework  was  developed  for 
identifying  and  presenting  a  number  of  recommendations  for  IPD  productivity 
improvement.  These  recommendations  are  presented  in  detail  in  section  III 
of  this  report.  The  major  functional  categories  for  the  series  of  recom¬ 
mendations  presented  are: 
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CNETS  memo  00  of  6  January  1975,  subj:  Concept  of  operations  for  imple¬ 
menting  centrally  managed  ISD;  development  of 
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CNET  Policy 

Organizational  Structure 
Management  and  Operations  Practices 
New  Technology 

Specific  issues  requiring  resolution  are  subsumed  under  each  of  these 
categories.  Recommendations  are  grouped  in  terms  of  these  specific  issues. 
Each  recommendation,  as  shown  below,  is  accompanied  by  supporting  rationale. 

CNET  Policy 

CNET  IPO  Role  in  Navy  Training  Plan  Development 
IPD  Prioritization  for  All  "A,"  "C,"  and  "F"  Courses 
IPD  Planning,  Programming,  and  Budgeting 
Assessment  of  IPD  Need  Prior  to  Development 
Resource  Expenditure,  Measurement,  and  Tracking 
Internal  Training  Effectiveness  Evaluation. 

Organizational  Structure 

IPDC  Organizational  Structure 
..  School  IPD  Organization 
..  CNET  IPD  Staff  Management  Organization. 

Management  and  Operations  Practices 

..  Planning  System 

Personnel  Selection,  Training,  and  Certification 

IPD  Needs  Determination 

IPD  Resource  Programming  and  Budgeting 

Course  Development/Maintenance  Prioritization  and  Tasking 

I PDC/C I SO/School  Program  Integration  and  Coordination 

IPD  Design,  Development,  and  Implementation 

Course  Level  Validation  and  Validation  Verification 

Training  Effectiveness  Evaluation 

Management  Information  System 

Productivity  Measurement  and  Evaluation  System. 

New  Technology 

A  post  note  to  the  study  presents  an  overview  of  policy  relating  to 
planning  and  management  control.  The  post  note  highlights  the  importance  of 
planning  and  demonstrates  the  interrelationships  and  interdependencies  among 
planning,  implementation,  measurement,  and  control. 
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SECTION  III 

RECOMMENDATIONS  AND  RATIONALE 

This  section  presents  a  series  of  recommendations  with  supporting 
rationale  for  enhancing  IPD  productivity.  These  measures  for  increasing 
program  effectiveness  are  intended  to  support  the  development  of  CNET  policy 
and  planning  for  the  efficient  implementation  of  a  command-wide  IPD  program. 
Because  of  the  nature  of  the  recommendations  presented  (wide-range  of  subject 
matter  affecting  many  functional  areas)  rapid  and  concurrent  implementation 
could  be  detrimental  to  development  programs  in  process.  This  possibility 
should  be  considered  during  the  development  of  a  long-range  plan.  The 
proposals  herein  are  presented  within  the  framework  developed  in  section  II 
of  this  report  (p.  6) . 

CNET  POLICY 

The  recommendations  presented  relate  to  CNET  long-range  goals  and 
objectives  for  the  ISD  program.  The  underlying  objective  for  the  issues 
addressed  herein  is  to  support  these  goals  and  objectives  by  modifying 
current  policy  and  guidance  to  be  more  explicit  and  relevant. 

CNET  IPD  ROLE  IN  NAVY  TRAINING  PLAN  DEVELOPMENT.  Establish  policy  which 
assures  CNET  IPD  personnel  participation  in  Navy  Training  Plan  development. 

During  the  Navy  Training  Plan  development  process  a  conceptualization 
takes  place  which  provides  a  basis  for  establishing  future  training  system 
requirements.  The  training  systems  developed  to  satisfy  these  requirements 
must  be  supportive  of  the  weapons  systems  to  which  they  relate  and  should 
utilize  state-of-the-art  training  technology.  The  increasing  sophistication 
of  weapon  system  and  training  system  technology  requires  professional  exper¬ 
tise  in  the  decision-making  process.  This  expertise  is  being  provided  at  the 
weapons  system  development  level.  However,  it  is  often  overlooked  at  the 
training  plan  development  level,  due  to  differences  in  the  perceived  im¬ 
portance  of  operational  systems  and  training  systems. 

Establishing  a  requirement  for  IPDC  instructional  technologists  to 
participate  in  training  plan  development  would  assure  representation  by 
training  professionals.  It  would  also  provide  to  these  professionals  an 
appreciation  of  Fleet  training  problems  relating  to  new  weapon  systems. 
Additionally,  it  would  serve  to  identify  prerequisite  core  training  require¬ 
ments  relating  to  the  training  plan  under  development.  Although  scarce 
personnel  resources  would  be  required  to  take  this  recommended  action,  it  is 
proposed  that  the  benefit  gained  would  more  than  offset  the  cost  of  resources 
expended. 

IPD  PRIORITIZATION  FOR  ALL  "A,"  "C,"  AND  "F"  COURSES.  Establish  a  single  IPD 
priori tization  policy  for  all  "A,"  "C,"  and  "F"  courses.  This  prioritization 
policy  would  insure  a  balance  between  economic  factors  and  critical  need 
factors  in  the  selection  of  courses  for  development.  It  would  also  provide  a 
basis  for  including  all  CNET  training  programs  in  the  prioritization  process. 
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During  the  process  of  allocating  scarce  resources  to  IPD  projects,  it 
becomes  necessary  to  prioritize  development  activity  on  the  basis  of  benefit 
and  cost.  The  prioritization  system  which  has  evolved  for  this  purpose 
appears  to  be  weighted  heavily  in  favor  of  return  on  investment.  If  this 
return  were  measured  in  terms  of  Navy  effectiveness,  a  problem  would  not 
exist.  However,  this  is  not  the  case.  Under  the  existing  system  it  is 
likely  that  a  high  throughput  "A"  School  would  be  selected  for  IPD  over  a 
critical  need  "C"  or  "F"  School.  This  might  or  might  not  lead  to  greater 
Navy  effectiveness  since  training  criticality  is  not  considered. 

All  "A,"  "C,"  and  "F"  courses  require  periodic  program  development  and 
maintenance  and  most  are  under  CNET  cognizance.  However,  there  are  not 
sufficient  development  and  maintenance  resources  to  apply  to  all  3,000+ 
courses  requiring  support,  and  it  is  likely  that  this  situation  will  not 
change  in  the  near  term.  Consequently,  a  need  exists  to  establish  a  policy 
which  will  assure  a  periodic  curriculum  effectiveness  review  of  all  Navy 
training  under  CNET  cognizance  to  assure  proper  allocation  of  limited  re¬ 
sources.  A  prioritization  policy  requiring  a  periodic  review  of  all  training 
under  CNET  cognizance  should  help  to  alleviate  a  situation  in  which  critical 
need  development  is  often  overlooked. 

IPD  PLANNING,  PROGRAMMING,  AND  BUDGETING.  Develop  a  specialized  Planning, 
Programming,  and  Budgeting  System  (PPBS)  policy  for  IPD  programs  which  would 
allow  for  flexibility  in  the  programming,  budgeting,  and  allocation  of  IPD 
resources. 

Because  2-  to  3-year  lead  time  is  often  not  possible  in  the  IPD  process 
it  is  suggested  that  IPD  PPBS  policy  be  modified  to  allow  for  block  funding 
within  investment  appropriations.  Although  OPNAV  support  would  most  likely 
be  required  in  formulating  and  obtaining  approval  for  a  system  with  this 
flexibility,  an  action  of  this  type  is  considered  justifiable. 

The  PPBS  within  the  Department  of  Defense  (DOD)  has  provided  a  degree  of 
program  visibility  in  support  of  decision  making  which  did  not  exist  prior  to 
its  implementation.  For  this  reason,  PPBS  implementation  has  been  considered 
a  major  step  forward  in  DOD  program  management  and  control.  However,  when 
applied  in  support  of  IPD  within  CNET,  it  becomes  a  significant  obstacle  to 
the  efficient  execution  of  IPD  programs.  The  reason  for  this  situation  is 
primarily  one  of  projecting  IPD  need  2  to  3  years  in  advance  of  implemen¬ 
tation  and,  more  specifically,  one  of  identifying  types  of  training  equipment 
to  be  applied  prior  to  the  conduct  of  a  training  analysis.  It  has  been 
demonstrated  many  times,  both  by  CNTECHTRA  and  IPDCs,  that  programming  for 
investment  training  items  2  or  more  years  in  advance  of  need  is,  at  best,  an 
expert  guess.  Many  approaches  have  been  attempted  to  minimize  this  problem, 
but  none  have  been  successful.  The  best  was  the  development  of  generalized 
planning  wedges  which  allowed  for  entry  into  the  programming  process.  The 
artificiality  of  this  approach,  although  well  intended,  does  not  provide 
sufficient  justification  during  the  POM  review  process.  This  can  result  in 
greater  penalties  for  IPD  than  rewards. 

Because  of  the  nature  of  investment  appropriations  (OPN,  APN,  SCN),  CNET 
cannot  program  for  block  funding  in  these  categories.  It  is,  therefore, 
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suggested  that  action  be  taken  at  the  OPNAV  level  to  explore  programming 
alternatives  for  IPO  investment  items  in  order  to  alleviate  a  problem  which 
could  become  very  serious. 

ASSESSMENT  OF  IPD  NEED  PRIOR  TO  DEVELOPMENT.  Establish  policy  for  determining 
and  documenting  IPD  need  at  the  course  level  prior  to  initiating  design  and 
development  activity. 

The  experience  gained  to  date  on  Navy  IPD  programs  indicates  that 
resource  expenditure  per  hour  of  instructional  program  development  is  signi¬ 
ficant.  It  is  also  noted  that  total  course  development  tasking  is  initiated 
without  first  assessing  if  only  a  portion  of  a  curriculum  (such  as  a  phase  of 
instruction)  is  deficient  and  in  need  of  development.  This  sometimes  results 
in  curriculum  development  involving  hundreds  of  hours  of  instruction  as 
opposed  to  development  of  a  phase  which  might  involve  50  hours  or  less.  The 
continuation  of  this  tasking  and  development  approach  could  result  in  deve¬ 
lopment  expenditures  greatly  in  excess  of  actual  need. 

The  course  of  action  recomnended  above  would  require  an  assessment  and 
documentation  of  development  need  prior  to  tasking  for  design  and  develop¬ 
ment.  In  essence,  this  policy  would  assure  that  resources  are  utilized  to 
make  bad  instruction  good  as  opposed  to  making  good  instruction  better.  In 
this  way  critical  need  will  be  identified  and  addressed  prior  to  noncritical 
need. 

RESOURCE  EXPENDITURE,  MEASUREMENTS  AND  TRACKING.  Install  procedures  for 
measuring  and  tracking  resources  expended  during  all  instructional  program 
development  and  maintenance  activity  within  the  CNET  command. 

The  expenditure  of  CNET-control led  resources  for  IPD  is  not  readily 
determinable.  A  resource  expenditure  and  accountability  system  for  IPDC 
programs  was  implemented  in  19764  and,  although  many  imperfections  are 
apparent  in  this  system,  it  provides  reliable  information  on  a  limited  scale. 
Progress  has  been  made  in  the  development  of  IPD  management  systems  since  the 
implementation  of  formalized  IPD  activity  within  the  CNET  command,  and 
management  control  systems  continue  to  improve.  However,  resource  expendi¬ 
ture  is  not  controlled  effectively  because  a  corranand-wide  IPD  resource  expendi¬ 
ture  tracking  system  does  not  exist.  IPD  activity  is  somewhat  unique  and  the 
differentiation  between  IPD  activity  and  other  activity  within  the  Command  is 
not  readily  determinable.  This  makes  measurement  very  difficult.  In  addi¬ 
tion,  there  is  some  strong  sentiment  that  resource  expenditure  measurement 
and  tracking  for  IPD  would  cost  more  than  its  worth.  This  line  of  reasoning 
does  not  appear  justifiable.  The  magnitude  of  IPD  expenditures  and  the 
productivity  levels  being  achieved  indicate  that  an  effective  measurement  and 
tracking  system  could  be  of  significant  benefit. 


^CNET  SUPPORT  Instruction  7400.1  of  27  June  1978,  subj:  Man-Hour  Accounting 
for  IPD;  Policy  and  Procedure  for 
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RNAL  TRAINING  EFFECTIVENESS  EVALUATION.  Conduct  internal  Training 
ctiveness  Evaluation  (TEE)  for  selected  training  programs  developed  in 
rdance  with  ISO  guidelines.  The  selective  application  of  this  TEE 
bility  would  be  used  in  conjunction  with  resource  expenditure  information 
evelop  benefit- to-cost  measures. 

All  of  the  CNET  3,000+  training  courses  were  developed  in  accordance 
some  systematic  development  process  and  all  require  periodic  update 
ng  their  life  cycle.  This  development  and  maintenance  activity  accounts 
significant  resource  expenditures,  the  magnitude  of  which  has  not  been 
ized  until  recent  years.  The  formalization,  control,  and  accountability 
he  instructional  program  development  process  within  CNET  has  provided  a 
ted  information  base  upon  which  I  PD  resource  allocation  decisions  are 
.  However,  there  is  one  essential  element  of  management  information 
nt--a  measure  of  training  program  effectiveness .  CNET  policy  does  not 
ire  a  pre-  and  post-IPD  effectiveness  evaluation.  Without  this  measure 
raining  effectiveness,  it  is  not  known  if  an  updated  course  is  less 
ctive  than,  equal  to,  or  more  effective  than,  the  original  course  in 
sfying  training  objectives.  Consequently,  it  is  not  possible  to  develop 
nagement  control  system  which  will  assure  efficient  allocation  of  re- 
ces  in  support  of  training  effectiveness  improvement. 

The  policy  recommended  above  would  allow  for  the  selective  application 
nternal  TEE  to  the  extent  that  it  would  be  supportive  of  the  IPD  mission, 
ough  the  ultimate  evaluation  is  in  the  operational  environment,  it  is  not 
ested  that  IPD  TEE  be  extended  to  this  level.  The  TEE  recommended  above 
tes  to  the  effectiveness  of  training  provided  as  measured  in  the  school 
he  completion  of  training.  This  would,  in  essence,  certify  that  the 
ted  course  of  instruction  is  effective  in  satisfying  the  course  training 
ctives  to  a  greater  or  lesser  degree  than  the  original  course.  This 
d  then  provide  a  basis  for  management  action  to  assure  IPD  training 
em  cost  effectiveness. 

NIZATIONAL  STRUCTURE 

The  organizatii  ' ’  structure  recommendations  contained  in  this  section 
evolved  from  the  •'am  concepts  in  Hodak,  Middleton,  and  Rankin 
9),  the  need  to  redu>  head  as  identified  in  Swope  and  Keeler  (1981), 

the  many  organizational  recommendations  offered  by  key  IPD  personnel. 

The  organizational  structures  proposed  in  this  report  are  not  intended 
e  in  a  form  ready  for  staffing.  They  are  proposed  only  as  conceptual 
nization  models  intended  to  be  consistent  with  other  IPD  recommendations 
as  team  structuring,  matrix  support,  reduction  of  indirect  charges,  and 
r  product! vi ty-related  measures. 

ORGANIZATIONAL  STRUCTURE.  The  model  IPDC  structure  (figure  1)  is  an 
nizational  guideline  for  Navy  IPDC  organizational  development. 

The  organizational  structure  shown  in  figure  1  provides  many  of  the 
rable  characteristics  of  an  IPD  organization  as  identified  in  Hodak, 
leton,  and  Rankin  (1979)  and  Swope  and  Keeler  (1981).  There  is  a  single 
team  management  function  subordinate  to  the  IPDC  director,  and  team 
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Figure  1.  IPOC  Organizational  Structure 
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leaders  are  participating  educational  technologists.  This  organizational 
arrangement  reduces  layering  of  management  and  provides  project  team  leaders 
with  the  necessary  authority  and  responsibi ltiy  for  ^velopment  programs 
under  their  cognizance. 

The  matrix  approach  to  providing  support  services  allows  for  support  of 
many  concurrent  programs  and  for  effective  management  planning  and  control. 
This  support  concept  can  also  be  extended  to  provide  support  services  for 
school-managed  IPD  programs  as  described  in  the  School  IPD  Organization 
section  of  this  report. 

The  new  technology  integration  and  evaluation  section,  which  would 
typically  consist  of  one  to  five  or  six  personnel,  will  provide  new  techno¬ 
logy  expertise  during  the  introduction  of  new  applications  such  as  comput¬ 
erized  typesetting,  computer  based  instructional  delivery,  and  computer  aided 
authoring.  It  is  intended  that  the  new  technology  specialists  would  integrate 
with  a  development  team  in  new  technology  implementation.  In  this  way  the 
risk  normally  associated  with  new  technology  applications  should  be  signi¬ 
ficantly  reduced. 

The  administration  and  quality  assurance  staff  functions  would  not 
change  significantly.  The  only  change  recommended  is  for  the  evaluation 
function  of  the  quality  assurance  office  to  be  limited  to  validation  verifi¬ 
cation.  Validation  should  be  a  team  leader  responsibility  with  third  party 
verification  a  quality  assurance  responsibility.  Under  this  operational 
arrangement,  a  small  one  or  two  person  evaluation  office  could  support  many 
development  programs. 

SCHOOL  IPD  ORGANIZATION.  Utilize  the  proposed  school  IPD  organizational 
structure  (figure  2)  as  an  organizational  guideline  in  developing  school- 
based  IPD  capabil ities. 

The  school  normally  participates  in  instructional  program  development 
conducted  at  IPDCs.  However,  the  IPDCs  do  not  normally  actively  participate 
in  development  or  course  update  projects  accomplished  at  the  sshool  by  school 
personnel.  The  school  IPD  organization  shown  in  figure  2  would  provide  for  a 
formalization  of  school  managed  IPD  using  support  services  provided  by  the 
IPDC.  These  support  services  would  be  provided  in  the  same  way  they  are  for 
IPDC  projects.  Consequently,  there  would  not  be  a  requirement  for  IPD  re¬ 
structuring.  Most  likely,  there  will  always  be  a  problem  of  insufficient 
school  and  IPDC  resources,  but  this  organizational  approach  should  help 
alleviate  the  problem  within  the  limit  of  available  resources.  There  is  a 
continuing  need  to  maintain  currency  for  all  CNET  training  programs.  The 
school  IPD  organization  shown,  supported  by  IPDCs,  would  allow  for  improved 
responsiveness  to  this  requirement. 

CNET  IPD  STAFF  MANAGEMENT  ORGANIZATION.  Utilize  the  structure  (figure  3)  as 
an  organizational  guideline  in  configuring  CNET  staff  management  in  support 
of  command-wide  IPD. 

The  proposed  CNET  IPD  staff  management  organization  shown  in  figure  3  is 
very  similar  to  the  current  organization.  The  only  changes  suggested  relate 


School  IPD  Organization 
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to  greater  organizational  emphasis  on  policy,  standards,  planning,  and  evalu¬ 
ation  and  to  formalizing  Training  Functional  Program  Coordination.  IPD 
personnel  have  suggested  that  CNET  policy  and  standards  capabilities  should 
be  expanded  and  operations  management  functions  should  be  reduced.  The 
organizational  structure  shown  satisfies  this  goal. 

Implementing  a  command-wide  IPD  policy  will  require  increased  CNET/ 
training  functional  IPD  interface,  particularly  for  "C"  and  "F"  course 
development.  The  organization  shown  provides  a  Training  Functional  Coordina¬ 
tion  section  for  this  purpose. 

Other  staff  functions  remain  essentially  the  same.  It  is  not  envisioned 
that  CNET  staff  expansion  would  be  necessary  for  the  organization  shown 
beyond  the  one  or  two  specialized  billets  needed  for  added  organizational 
functions. 

MANAGEMENT  AND  OPERATIONS  PRACTICES 

The  recommendations  for  improving  management  and  operations  practices 
discussed  in  this  report  were  initially  described  in  Hodak,  Middleton,  and 
Rankin  (1979)  and  Swope  and  Keeler  (1981).  Based  upon  these  suggestions  for 
productivity  enhancement,  comments  were  solicited  from  key  IPD  personnel 
regarding  feasibility.  The  responses  indicated  that  most  of  the  recommenda¬ 
tions  appeared  feasible  and  that  some  were  already  under  consideration  for 
implementation. 

The  recommendations  proposed  in  the  remainder  of  this  section  relate  to 
operating  procedures  and  address  the  need  to  prepare  instructions,  publish 
handbooks,  and  establish  standard  practices  which  relate  to  and  support  CNET 
IPD  policy.  Following  each  recommendation,  related  comments  are  provided 
which  highlight  content  areas  wherein  productivity  improvement  is  possible. 

PLANNING  SYSTEM.  A  number  of  plans  have  been  developed  for  IPD  since  the 
approval  of  the  original  master  plan  in  August  1975.  These  plans  have 
included  long-range,  mid-range,  and  short-range  plans  for  approximately  25 
instructional  program  development  projects  and,  additionally,  many  IPDC 
support  projects.  Because  there  is  little  consistency  among  the  plans  in 
terms  of  content  or  format,  each  project  plan  is  essentially  unique. 

It  is  suggested  that  a  standardized  planning  system  be  developed  and 
procedural i zed  for  command-wide  IPD  which  could  be  used  effectively  to 
support  resource  programming  and  management  control.  This  system  would 
incorporate  global  or  master  planning  at  the  CNET  level,  mid-range  planning 
at  the  IPDC  level,  and  short-term  project  planning  at  the  project  level. 

This  integrated  planning  system  should  be  supportive  of  CNET  master  planning 
objectives,  and  each  planning  level  should  provide  for  the  rapid  incorpora¬ 
tion  of  approved  changes. 

PERSONNEL  SELECTION,  TRAINING,  AND  CERTIFICATION.  Personnel  selection  and 
training  are  necessary  functions  in  support  of  formalized  IPD  within  CNET. 
However,  standards  and  procedures  specifically  addressing  these  IPD  functions 
have  not  been  developed.  Therefore,  it  is  suggested  that  an  instruction  be 
developed  and  promulgated  for  all  IPD  occupational  series  specifying  selection 
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criteria,  training  requirements,  and  certification  standards.  Although  an 
instruction  of  this  type  will  not  bring  about  short-term  improvement,  it  will 
provide  a  baseline  document  from  which  a  personnel  selection,  training,  and 
certification  system  can  evolve. 

I  PD  NEEDS  DETERMINATION.  The  requirement  for  I  PD  needs  assessment  is  becoming 
increasingly  evident.  Tasking  and  management  control  cannot  be  effective  if 
IPD  need  is  unknown.  When  the  results  of  preanalysis,  which  specifies  what 
should  be  trained,  are  compared  to  the  curriculum  which  describes  what  is 
being  trained,  a  training  deficiency  (need)  or  training  excess  (no  need) 
results.  This  need  should  be  documented  prior  to  design  and  development 
tasking  for  effective  program  control.  It  is  suggested  that  a  procedure  for 
this  process  be  developed  and  distributed  for  use  on  all  IPD  programs  within 
the  CNET  command. 

IPD  RESOURCE  PROGRAMMING  AND  BUDGETING.  The  DOD  PPBS  has  proven  its  worth  as 
an  effective  management  planning  and  programming  system  for  weapons  system 
planning  and  acquisition.  It  can  also  be  used  effectively  for  any  type  of 
PPB  activity  in  which  long-range  needs  can  be  identified  and  specified. 
However,  when  applied  in  support  of  IPD  projects,  it  can  severely  restrict 
the  IPD  options  available  to  the  development  agent.  This  topic  has  already 
been  discussed  in  this  report  under  CNET  policy  recommendations. 

The  IPD  resource  planning,  programming,  and  budgeting  procedure  addressed 
here  is  intended  to  provide  a  systematic  method  for  identifying  long-term 
needs  for  future  IPD  programs.  With  this  assessment  of  future  need,  gen¬ 
eralized  planning  wedges  can  be  entered  during  the  POM  process  to  be  further 
defined  during  the  budget  process.  Although  this  generalized  submission  is 
not  desirable,  it  is  the  only  available  option  within  the  current  system. 
Consequently,  it  should  be  procedural i zed  and  documented  for  use  by  all  IPD 
activities.  It  does  not  provide  a  long-term  solution  to  the  problem  but  it 
should  provide  a  limited  means  of  replacing  and  adding  to  the  capital  base  of 
training  equipment  which  is  in  a  serious  state  of  erosion. 

COURSE  DEVELOPMENT/MAINTENANCE  PRIORITIZATION  AND  TASKING.  The  primary 
factor  considered  to  date  in  the  prioritization  of  IPDC  projects  is  course 
throughput.  If  student  hours  trained  and  payback  were  the  only  factors  of 
importance  to  the  Navy,  this  might  be  the  preferable  way  to  continue. 

However,  this  approach  would  lead  to  satisfactory  development  and  maintenance 
of  large  throughput  courses  such  as  "A"  schools,  but  smaller  throughput 
courses  such  as  "C"  and  "F"  schools  would  eventually  become  seriously  out¬ 
dated. 

The  suggested  solution  to  this  problem  is  to  develop  and  publish  a 
prioritization  and  tasking  procedure  for  all  courses  under  CNET  cognizance. 
This  procedure  should  consider  such  factors  as  course  criticality,  impact  on 
fleet  effectiveness,  time  since  last  revision,  and  other  related  non-through¬ 
put  considerations.  This  prioritization  process  should  be  managed  by  a 
single  office,  and  an  interface  should  be  maintained  with  all  training  func¬ 
tionals  during  the  prioritization  process.  Prioritization  in  an  environment 
of  limited  resources  is  difficult  and  a  system  for  maintaining  balance  in 
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resource  allocation  is  even  more  difficult.  However,  some  systematic  priori¬ 
tization  procedure  is  required  if  there  is  to  be  an  equitable  distribution  of 
I  PD  resources  throughout  the  command. 

I PDC/C IS0/SCH00L  PROGRAM  INTEGRATION  AND  COORDINATION.  During  the  execution 
of  an  IPD  project  there  is  a  continuing  requirement  for  IPDC  and  school 
personnel  interface  and  coordination.  This  requirement  is  equally  valid  for 
IPDC-managed  programs  or  school -managed  IPD  programs  supported  by  an  IPDC. 
Currently,  memos,  letters  of  understanding,  and  instructions  address  portions 
of  this  requirement  for  specific  phases  of  IPD  (SME  support,  programming  and 
budgeting,  life  cycle  support).  However,  it  is  suggested  that  a  single  CNET 
instruction  be  prepared  which  identifies  the  coordination  required  for  IPD 
projects.  This  instruction  would  also  specify  coordination  practices  to  be 
followed.  It  is  likely  that  an  instruction  of  this  nature  would  lead  to  a 
more  efficient  integration  and  utilization  of  command-wide  IPD  resources. 

IPD  DESIGN,  DEVELOPMENT,  AND  IMPLEMENTATION.  Procedures  for  IPD  design, 
development  and  implementation  were  initially  promulgated  within  CNET  as 
NAVEDTRA  110  (1978).  These  procedures  were  intended  to  be  an  extension  of 
NAVEDTRA  106A  (1975)  and  were  to  prescribe  CNET  policy,  procedures,  and 
guidelines  for  the  analysis,  design,  and  development  of  NAVEDTRACOM  instruc¬ 
tional  programs.  These  procedures,  although  very  useful  as  a  supplement  to 
NAVEDTRA  106A,  did  not  sufficiently  describe  IPD  practices  to  be  followed  at 
the  working  level.  Consequently,  CNET  (N-9)  initiated  a  program  for  the 
restructuring  and  update  of  this  document.  This  effort  is  scheduled  to  be 
completed  during  1981. 

CdJRSE  LEVEL  VALIDATION  AND  VALIDATION  VERIFICATION.  During  the  normal 
processes  involved  in  program  development,  there  is  a  requirement  to  validate 
new  or  updated  instructional  material  in  terms  of  requisite  training  objec- 
t  ves.  This  validation,  performed  by  the  development  team,  is  of  primary 
concern  to  three  parties--the  IPD  development  team,  IPDC  and  CNET  management, 
and  school  management.  In  order  to  provide  validation  assurance  to  these 
p.rties,  it  is  suggested  that  an  instruction  be  prepared  which  requires  a 
f(  rmal  validation  verification.  This  instruction  would  require  IPD  valida- 
t  on  verification  by  the  IPDC  evaluation  office  and  the  school  CISO.  Copies 
o  the  verified  validation  results  could  then  be  distributed  to  IPDC,  CNET, 
aid  school  management  advising  that  the  instructional  material  has  been 
v, lidated  and  is  suitable  for  training.  This  procedure  would  provide  the 
n< cess ary  assurance  to  CNET  and  school  management  that  revised  training 
programs  satisfy  the  training  objectives  developed  during  the  training  analysis 
pi  ase. 

TRAINING  EFFECTIVENESS  EVALUATION  (TEE).  For  selected  courses,  it  is  sug¬ 
gested  that  training  effectiveness  be  evaluated  before  and  after  IPD  treat¬ 
ment.  Without  this  evaluation,  it  is  difficult  to  determine  if  newly  deve¬ 
loped  instructional  material  is  superior  to  the  existing  instructional 
material.  The  ability  to  determine  improvement  in  training  effectiveness  is 
necessary  for  the  long-term  justification  of  IPD  resource  expenditure.  There¬ 
fore,  standardized  procedures  are  needed  for  this  evaluation  process.  It  is 
suggested  that  TEE  instructions  and  procedures  for  internal  evaluation  be 
prepared  and  promulgated  to  satisfy  this  requirement. 
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MANAGEMENT  INFORMATION  SYSTEM.  Although  management  information  is  currently 
provided  to  I  PD  managers,  a  number  of  comments  by  key  IPO  personnel  suggest 
that  the  information  is  inadequate  in  some  cases  and  not  needed  in  others. 
Since  managers  can  not  effectively  manage  without  feedback  information,  it  is 
suggested  that  two  actions  be  initiated.  The  first  would  consist  of  an 
analysis  of  management  information  needs.  If  it  is  demonstrated  that  certain 
information  is  not  necessary  for  management  decision  making  and  control,  it 
should  not  be  included  as  required  management  information.  Conversely,  if  a 
demonstrated  need  exists,  it  should  be  included  as  required  information. 

These  information  elements  should  be  identified  for  each  management  level 
from  team  leader  to  CNET  staff.  The  second  action  would  consist  of  the 
design,  development,  and  implementation  of  a  management  information  system 
which  would  provide  the  required  management  information.  This  information 
should  be  timely  and  in  a  format  suitable  for  use  at  all  management  levels. 

PRODUCTIVITY  MEASUREMENT  AND  EVALUATION  SYSTEM.  It  was  noted  in  Hodak, 
Middleton,  and  Rankin  (1979)  and  Swope  and  Keeler  (1981)  that  IPDC  produc¬ 
tivity  was  below  that  which  should  be  reasonably  expected.  Because  of  these 
findings,  the  present  study  was  specifically  directed  toward  identifying 
productivity  enhancement  measures. 

CNET  IPD  management  can  not  currently  track  productivity  effectively 
because  of  the  inadequacy  of  the  system  used.  It  is  proposed  that  a  pro¬ 
ductivity  measurement  and  tracking  system  be  developed.  It  is  also  proposed 
that  a  productivity  measurement  and  tracking  system  procedures  manual  be 
developed  and  published  for  command-wide  application. 

NEW  TECHNOLOGY 

The  productivity  enhancement  measures  discussed  in  prior  sections  of 
this  report  have  a  common  weakness.  They  address  policy,  organization,  and 
operations  issues  which  can  not  be  evaluated  in  a  quantifiable  form.  As  an 
example,  it  appears  that  decreasing  indirect  effort  and  increasing  direct 
effort  on  IPD  projects  would  enhance  productivity.  However,  there  is  not 
sufficient  evidence  to  state  with  confidence  that  this  will  be  the  case. 
Management  consensus,  management  information  data  analysis,  and  generally 
accepted  management  practices  have  been  utilized  in  developing  the  recom¬ 
mendations  herein  but  operational  implementation  and  evaluation  will  be  the 
only  true  test  of  productivity  enhancement. 

New  technology  measures  are  not  subject  to  this  same  uncertainty.  For 
most  cases  recommended,  there  have  been  numerous  applications  in  the  public 
and  private  sector  wherein  productivity  improvement  has  been  demonstrated. 

The  primary  risk  areas  in  the  application  of  new  technology  are  those  as¬ 
sociated  with  resistance  to  change,  selection  and  training  of  qualified 
personnel,  and  integration  with  existing  systems.  For  this  reason,  the  IPD 
organizational  structure  recommended  includes  an  organizational  function  for 
new  technology  integration  and  evaluation.  It  is  envisioned  that  an  organi¬ 
zational  unit  such  as  this  would  consist  of  personnel  with  advanced  technology 
experience  and  an  educational  background  in  instructional  technology,  experi¬ 
mental  psychology,  and/or  computer  systems  hardware  and  software  engineering. 
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The  new  technology  applications  being  considered  as  recommendations  are 
primarily  computer  based  because  this  is  currently  the  major  new  technology 
thrust.  These  recommendations  include  instructional  development,  instruc¬ 
tional  management,  and  instructional  delivery  applications  in  areas  such  as 
text/data  processing,  typesetting,  digitized  graphics,  authoring,  simulation, 
and  testing.  Because  of  the  scope  of  the  effort  required  for  the  identifi¬ 
cation,  specification,  and  justification  of  new  technology  initiatives,  and 
because  of  the  "stand  alone"  nature  of  the  subject  matter,  new  technology 
applications  should  be  addressed  as  a  separate  study.  An  outline  of  topics 
suggested  for  this  study  include: 

.  New  Technology  Integration  and  Evaluation  Section  at  Selected 

Location(s) 

.  Computer  Communications  and  Network  Standards 

.  Computerized  Data  Base  Management 

.  Computerized  Word/Text/Data  Processing 

.  Computerized  Typesetting 

.  Digitized  Graphics 

.  Computerized  Readability  Analysis 

Computer  Aided  Authoring 
.  Computerized  Instructional  Delivery 


Text 

Still  Graphic 

Animation 

Simulation 


Computerized  Interactive  Testing 
Computerized  Instructional  Systems  Management. 


POST  NOTE 

This  report  and  its  companion  report  (Swope  and  Keeler,  1981)  are  intended 
to  be  used  by  CNET  management  as  a  data  base  to  support  the  development  of 
long-term  CNET  IPD  strategy  and  goals.  This  CNET  effort  should  result  in  a 
long-range  master  plan  for  the  IPD  program  which  incorporates  "A,"  "C,"  and 
"F"  course  development.  It  should  also  incorporate  a  system  for  integrating 
the  planning  process  into  the  total  management  control  process. 
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The  operational  system,  at  the  IPDC  level,  should  be  designed  to  imple¬ 
ment  short-range  plans  which  are  separate  yet  related  subsets  of  the  master 
plan.  The  design  of  this  implementation  system  will  be  of  primary  importance 
in  the  achievement  of  greater  productivity.  The  policy,  organization,  and 
operations  recommendations  presented  in  this  report  should  result  in  produc¬ 
tivity  enhancement.  However,  many  are  long  range  in  nature.  For  short-range 
payoff,  it  is  suggested  that  initial  emphasis  be  placed  on  personnel  selection 
and  training,  the  application  of  production  management  techniques,  and  the 
implementation  of  a  system  for  output  measurement  and  tracking.  Although  a 
comprehensive  plan  should  assure  results  if  well  conceived  and  effectively 
implemented,  personnel  qualifications  and  management  control  mechanisms  are 
essential.  Deficiencies  in  these  areas  can  offset  potential  benefit  to  the 
degree  that  planning  serves  no  useful  purpose.  Consequently,  it  is  suggested 
that  planning  and  implementation  be  considered  as  complementary  activities 
during  the  development  of  policy  and  procedures  for  CNET  IPD. 
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